Measuring the Reliability of a Gamified Stroop Task: Quantitative Experiment.

BACKGROUND: Few gamified cognitive tasks are subjected to rigorous examination of psychometric properties, despite their use in experimental and clinical settings. Even small manipulations to cognitive tasks require extensive research to understand their effects. OBJECTIVE: This study aims to investigate how game elements can affect the reliability of scores on a Stroop task. We specifically investigated performance consistency within and across sessions. METHODS: We created 2 versions of the Stroop task, with and without game elements, and then tested each task with participants at 2 time points. The gamified task used points and feedback as game elements. In this paper, we report on the reliability of the gamified Stroop task in terms of internal consistency and test-retest reliability, compared with the control task. We used a permutation approach to evaluate internal consistency. For test-retest reliability, we calculated the Pearson correlation and intraclass correlation coefficients between each time point. We also descriptively compared the reliability of scores on a trial-by-trial basis, considering the different trial types. RESULTS: At the first time point, the Stroop effect was reduced in the game condition, indicating an increase in performance. Participants in the game condition had faster reaction times (P=.005) and lower error rates (P=.04) than those in the basic task condition. Furthermore, the game condition led to higher measures of internal consistency at both time points for reaction times and error rates, which indicates a more consistent response pattern. For reaction time in the basic task condition, at time 1, rSpearman-Brown=0.78, 95% CI 0.64-0.89. At time 2, rSpearman-Brown=0.64, 95% CI 0.40-0.81. For reaction time, in the game condition, at time 1, rSpearman-Brown=0.83, 95% CI 0.71-0.91. At time 2, rSpearman-Brown=0.76, 95% CI 0.60-0.88. Similarly, for error rates in the basic task condition, at time 1, rSpearman-Brown=0.76, 95% CI 0.62-0.87. At time 2, rSpearman-Brown=0.74, 95% CI 0.58-0.86. For error rates in the game condition, at time 1, rSpearman-Brown=0.76, 95% CI 0.62-0.87. At time 2, rSpearman-Brown=0.74, 95% CI 0.58-0.86. Test-retest reliability analysis revealed a distinctive performance pattern depending on the trial type, which may be reflective of motivational differences between task versions. In short, especially in the incongruent trials where cognitive conflict occurs, performance in the game condition reaches peak consistency after 100 trials, whereas performance consistency drops after 50 trials for the basic version and only catches up to the game after 250 trials. CONCLUSIONS: Even subtle gamification can impact task performance albeit not only in terms of a direct difference in performance between conditions. People playing the game reach peak performance sooner, and their performance is more consistent within and across sessions. We advocate for a closer examination of the impact of game elements on performance.

Tasting inhibition: A proof-of-concept study of the food stop-signal game.

Self-Control is an important skill in everyday life when attention is automatically drawn toward certain stimuli. For instance, food stimuli automatically capture visual attention and are processed preferentially. Therefore, efficient response inhibition is crucial to refrain from careless overeating. In the present proof-of-concept study we use a novel adaptation of a previously evaluated Stop-Signal Game (SSG) to measure reactive, food-specific, response inhibition in healthy adults. We analyzed a sample of 83 participants (60 female, mean age=24.1, mean BMI=21.71kg/m2) split into three groups. In a gamified stop-signal task, participants navigated an avatar in an urban environment toward high-calorie food, low-calorie food, or non-food stimuli in go-trials and were asked to inhibit the approach reaction in stop-trials. Hunger, eating styles, food craving, and impulsivity were assessed via self-reports to investigate their relationship with (food-specific) response inhibition. Results showed that response inhibition (in terms of stop-signal reaction time, SSRT) did not differ between the high-calorie, low-calorie, and non-food SSG which might be explained by characteristics of the sample. However, impulsivity was positively correlated with SSRT in the low-calorie SSG, whereas food-craving and hunger were positively related to response inhibition in the high-calorie SSG. Future studies could build upon the food SSG to measure and train food-specific response inhibition in the treatment of overeating.

Effective Gamification of the Stop-Signal Task: Two Controlled Laboratory Experiments.

BACKGROUND: A lack of ability to inhibit prepotent responses, or more generally a lack of impulse control, is associated with several disorders such as attention-deficit/hyperactivity disorder and schizophrenia as well as general damage to the prefrontal cortex. A stop-signal task (SST) is a reliable and established measure of response inhibition. However, using the SST as an objective assessment in diagnostic or research-focused settings places significant stress on participants as the task itself requires concentration and cognitive effort and is not particularly engaging. This can lead to decreased motivation to follow task instructions and poor data quality, which can affect assessment efficacy and might increase drop-out rates. Gamification-the application of game-based elements in nongame settings-has shown to improve engaged attention to a cognitive task, thus increasing participant motivation and data quality. OBJECTIVE: This study aims to design a gamified SST that improves participants' engagement and validate this gamified SST against a standard SST. METHODS: We described the design of our gamified SST and reported on 2 separate studies that aim to validate the gamified SST relative to a standard SST. In study 1, a within-subject design was used to compare the performance of the SST and a stop-signal game (SSG). In study 2, we added eye tracking to the procedure to determine if overt attention was affected and aimed to replicate the findings from study 1 in a between-subjects design. Furthermore, in both studies, flow and motivational experiences were measured. RESULTS: In contrast, the behavioral performance was comparable between the tasks (P<.87; BF01=2.87), and the experience of flow and intrinsic motivation were rated higher in the SSG group, although this difference was not significant. CONCLUSIONS: Overall, our findings provide evidence that the gamification of SST is possible and that the SSG is enjoyed more. Thus, when participant engagement is critical, we recommend using the SSG instead of the SST.